The 360-degree video, also known as immersive video is an emerging technology, which can provide “feeling as sensation of present”. The sense of immersion is achieved by surrounding a user with wrap-around scene covering a panoramic view, in particular, 360-degree field of view. The “feeling as sensation of present” can be further improved by stereographic rendering. Accordingly, the panoramic video is being widely used in Virtual Reality (VR) applications.
Immersive video involves the capturing a scene using multiple cameras to cover a panoramic view, such as 360-degree field of view. The immersive camera usually uses a set of cameras, arranged to capture 360-degree field of view. Typically, two or more cameras are used for the immersive camera. All videos must be taken simultaneously and separate fragments (also called separate perspectives) of the scene are recorded. Furthermore, the set of cameras are often arranged to capture views horizontally, while other arrangements of the cameras are possible,
The 360-degree panorama camera captures scenes all around and the stitched spherical image is one way to represent the VR video, which continuous in the horizontal direction. In other words, the contents of the spherical image at the left end continue to the right end. The spherical image can also be projected to the six faces of a cube as an alternative 360-degree format. The conversion can be performed by projection conversion to derive the six-face images representing the six faces of a cube. On the faces of the cube, these six images are connected at the boundaries of the cube. In FIG. 1, image 100 corresponds to an unfolded cubic image with blank areas filled by dummy data. The unfolded cubic frame which is also referred as a cubic net with blank areas. As shown in FIG. 1, the unfolded cubic-face images with blank areas are fitted into a smallest rectangular that covers the six unfolded cubic-face images.
These six cube faces are interconnected in a certain fashion as shown in FIG. 1 since these six cubic faces correspond to six pictures on the six surfaces of a cubic. Accordingly, each boundary on the cube is shared by two cubic faces. In other words, each four faces in the x, y and z directions are continuous circularly in a respective direction. The circular boundaries for the cubic-face assembled frame with blank areas (i.e. image 100 in FIG. 1) are illustrated by image 200 in FIG. 2. The cubic boundaries associated with the cubic face boundaries are labelled, The cubic face boundaries with the same boundary number indicate that the two cubic face boundaries are connected and share the same cubic edge. For example, boundary #2 is on the top of face 2 and on the right side of face 4. Therefore, the top of face 2 is connected to the right side of face 4. Accordingly, the contents on the top of face 2 flow continuously into the right side of face 4 when face 2 is rotated 90 degrees counterclockwise.
While FIG. 1 and FIG. 2 illustrate an example of unfolded cubic net and cubic boundary connectivity for a cubic face representation of 360-degree virtual reality (VR) video, there also exist other multi-face representations of 360-degree VR video. The multiple face representation can be separated into multiple faces or a multi-face frame. Each sequence associated with one face can be coded separately. Alternatively, the frame sequence can be coded as a video sequence, where each frame corresponds to one unfolded multi-face frame or one assembled frame from the multiple faces. In the present invention, generation and usage of Inter coding references are disclosed.